Remote desktop technology allows a user of a client computing device to interact with a graphical user interface (e.g., a desktop interface) that is managed by a server computing device. The graphical user interface (GUI) includes display information to be presented on a display for a user. For example, display updates generated by software applications executed by the server may be rendered to a framebuffer at the server and then transmitted to the client for display. The framebuffer stores pixel color values for every pixel for an output display. Remote desktop and computing applications export the GUI to a remote device, referred to herein as the “client device.” Conversely, user inputs (e.g., mouse clicks and keystrokes) at the client device may be transmitted to the server for input into applications or other software running at the server.
To limit the amount of data transferred between the server and client, the GUI may be encoded. For example, the server may compress every frame and/or calculate differences between a current rendering (e.g., frame) of the desktop interface and a previous rendering, and transmit the differences to the client, rather than transmitting the entirety of the desktop interface.
Further, the desktop interface may be “virtual,” generated by a virtual machine (VM) executed by the server. In such virtual desktop infrastructures (VDIs), the GUI may be rendered by a virtual graphics processing unit (VGPU) stack associated with the VM, encoded, and transmitted to the client for display. When a video stream provided in encoded form is to be displayed within the desktop interface, the VGPU decodes the encoded video stream and renders the decoded video stream within the desktop interface. The desktop interface, including the decoded video stream, is then encoded and transmitted to the client.